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The importance of facilitating critical reasoning in the new 
millennium: some new evidence 

Bob Pithers 



Abstract 

Critical reasoning is seen to be a generic, transferable entity and an important outcome of 
most tertiary education courses currently available. This study investigated the critical 
reasoning skills of a group of Australian adult tertiary students who were practising adult 
vocational and further education teachers and trainers, using the recently developed Critical 
Reasoning Test (CRT). It examined some variables that might be expected to affect critical 
reasoning scores. These tertiary teachers’ mean CRT scores were found to be only at or 
below the “average” levels of other professional, non-teacher normative samples. No 
significant between-group CRT mean differences were found for these students’ Year of 
Course, Graduate/Non-graduate status, Age or Gender. In conclusion, the effective 
teaching-learning of the skills and dispositions underlying critical reasoning does not 
appear to be progressing well in at least some areas of tertiary education. Evidence such as 
this suggests that more attention should be given to facilitating this important area of 
student learning in universities in the new millennium. 

Introduction 

Critical reasoning is usually seen to be an important skill that needs to be taught at all levels 
of the educational systems in Australia, Great Britain and North America. In Australia, for 
instance, this skill has come to be seen as one of the major generic competencies (Mayer, 
1992); capable of generalisation across a broad range of educational contexts and work 
environments. It is government policy that teachers in the area of tertiary education as well 
as in other areas of education, should teach and facilitate their students ‘good’ thinking or 
reasoning skills to enable them to solve the problems they will face at work and in life. An 
important component of this scenario is that the teachers themselves should be capable of 
critical reasoning. Furthermore, that they should be able to teach this skill-based 
competency to their students. 

Critical reasoning may be an essential ingredient of effective teaching in the new 
millennium, especially for graduates who move to workplaces where they are exposed to 
much technological, social, economic and workplace change; where they are expected to 
think “smarter” than before. Very little appears to be known, however, about whether 
tertiary teachers have and are able to use the critical reasoning skills that they are expected 
to teach. Worldwide the assessment and evaluation of critical thinking it has been noted has 
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been neglected, although there has been attention directed at its teaching and learning in 
some published material (eg., Kennedy, Fisher and Ennis, 1991). A readily accepted 
definition, together with the appropriate assessment and evaluation of critical thinking 
remains problematic. Problems of practical difficulty also involve how to validly, reliably, 
time- and cost-effectively measure the concept critical reasoning or thinking. 

The term as it is currently used in the research literature, often broadly reflects Ennis’ 
(1993) definition as reasonable, reflective thinking focussing on action or belief This is a 
definition which excludes creative thinking. Conceived of in this fashion, critical thinking 
is composed of dispositions and skills or abilities. Attitudes or dispositions are affective, 
whilst skills are seen to be metacognitive in nature. These sets of factors appear to be 
involved in identifying the problem and assumptions that surround it, clarifying and 
focussing, understanding and applying inference, inductive and deductive logic, as well as 
judging the validity and reliability of sources of information (Kennedy et al., 1991). 

Ennis’ (1993) idea of critical thinking or reasoning, involved broad components which are 
seen to be generalisable across various domains; components such as “weighing the 
credibility of evidence”, being “open minded”, and “drawing warranted conclusions 
cautiously”. Others have argued, however, that critical reasoning is subject specific with 
each subject having its own type of reasoning (eg., McPeck, 1990). In practice, various 
evaluation instruments have been used to assess critical thinking and reasoning such as 
reflective-judgement scales, self-report questionnaires, diary reports, interview scales and 
various performance tests. One reasonably recent standardised test, developed by Smith 
and Whetton (1992), however, does appear to have promise for the examination of critical 
reasoning with adults in the area of adult, tertiary education. This is the Smith-Whetton 
Critical Reasoning Test (CRT), which claims to assess vocationally relevant critical 
reasoning skills. The theoretical conceptions underpinning the CRT are clearly linked with 
Ennis’s definition of critical thinking. The aspects of critical reasoning tested in the CRT, 
are the skills of analysis, evaluation, planning, valuing evidence and the dispositions of 
thinking critically and decision making on the basis of evidence and reasons. These factors 
broadly correspond to the skills and dispositions outlined by Ellis as the major ingredients 
of critical thinking. 

The CRT, therefore, was used to measure the thinking skills of a group of tertiary teachers 
and trainers in the present study. It was assumed that tertiary teachers with stronger 
critical thinking skills might be at an advantage when it came to teaching these skills. It 
was hypothesised that a group of selected tertiary teachers would have a significantly 
higher level of critical reasoning skills than the more general CRT non-teacher norms; 
secondly, that graduate students would have a significantly higher CRT mean score than 
non-graduates; thirdly, that Stage 2 or 3 students would have a significantly higher mean 
CRT score than Stage 1 students, that is given that University type courses do develop 
generic critical thinking and lastly, it was thought that there might be no significant CRT 
mean score differences in the sample because of gender or age level. 
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Method 



Sample 

The sample was composed of 1 1 1 teachers of whom 53 were women and 58 were men. All 
of these individuals were at a university studying for an educational qualification at degree 
or post-graduate diploma level. Their ages ranged from 24 to 57 years with a mean age of 
38 years. There were 49 individuals in Stage 1 of their university education course and 62 
individuals in Stages 2 or 3. Of the total sample, 89 teachers were non-graduates whilst 22 
were university graduates in disciplines other than education. All of the non-graduates held 
an appropriate trade or business certificate or diploma and were completing an 
undergraduate degree in education. The graduate students all held a degree in a domain 
other than education. For ethical reasons all of the subjects who made up this sample of 
convenience were all volunteers. They were all teachers in adult and further education 
whose experience averaged approximately four years. 

Instrument 

The Critical Reasoning Test (CRT) is composed of two major subtests: Verbal and 
Numerical. The CRT Verbal subtest (Australian Verbal edition) was used in this study. It 
purports to measure the intellectual skills relevant to the processing and interpretation of 
new information (Smith and Whetton, 1992). The test also includes questions which 
require the testee to recognise that insufficient information has been provided for a 
definitive judgment, similar to many real-life situations. The CRT Verbal subtest consists 
of 56 questions which must be completed in 20 minutes. The questions all relate to 
scenarios in which two trainee junior managers within a fictitious company are faced with 
various situations requiring critical reasoning. 

The Verbal subtest is divided into three sections. The first section, Analysis, requires 
information to be derived from a formal document provided. Evaluation, the second 
section, requires new information to be assessed in terms of its significance for current 
beliefs. The third section, Assumptions, asks the test taker to read a number of comments 
and assess the statements which follow, deciding whether or not they indicate an 
assumption held by the person making the comment. The CRT authors reported internal 
consistency reliability estimates ranging from .67 to .78 for the Verbal scale. The British 
normative representative sample presented in the manual is based on samples of 353 and 
143 people, further education college students (in 16 Colleges) and first-level managers, 
respectively. This normative sample was used for comparative purposes as their are no 
Australian norms presently available. 

Procedure 

The test was administered in small groups of 5 to 15 individuals according to the 
recommended test procedure in the manual. Participants were given six minutes for each of 
Sections I and II and eight minutes for Section III. 
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Results 



The overall sample mean score (comparative normative sample data in brackets) was 33.8 
(33.9); the standard deviation was 6.3 (6.7). A two-factor Analysis of Variance (ANOVA) 
applied to Year of Course (for non-graduates only; Stage 1 vs Stages 2/3) and Level of 
academic qualification (Graduate vs Non-graduate) indicated no significant main effect for 
either factor (F=1.24,/?>.05; F=3.00,/?> 05, respectively; Alpha level was set at .05 for all 
tests of significance). The interaction effect was also not significant (F=.4,/?>.05 ). 

The results of the analysis also revealed that there was neither a significant main effect for 
Gender (F=.9,/?>.05) nor a significant interaction effect (F=.36,/?>.05). 

Discussion 

It was found using the Smith- Whetton Critical Reasoning Test (CRT) that, on average, the 
tertiary teachers in the sample only scored at an “average” level (ie., 49th percentile, see 
Smith & Whetton, 1992) when compared to non-teacher norms based on first-line non- 
graduate retail managers. It was predicted that experienced tertiary adult vocational 
teachers, including some at degree level, all attempting an education degree course or a 
post-graduate qualification in education would have had a comparatively higher relative 
mean critical reasoning score. There were no significant differences between the mean 
CRT scores for graduates compared to non-graduates. Nor were there any significant 
differences found for gender or age. 

Graduate tertiary teachers or so it had been expected, should have had significantly higher 
mean scores on the CRT than non-graduates. That is if in a graduate education the sort of 
multi-aspect, broad-based metacognitive reasoning skills proportedly measured by the CRT 
were taught and learned. Another outcome of this study was that there was no significant 
difference between the samples’ CRT mean scores based on year (Stage) of present 
education degree course. A significant effect had been predicted here, based on the 
assumption that the course development of generic critical reasoning competencies ought to 
underlie the teachers’ cognitive development. Certainly an important assumption of the 
work of Ennis (1993), Smith and Whetton (1992), Sodden (1994) and other workers in this 
area is that the sort of abilities and dispositions reflected in the CRT are leamable. 

It was noted that the teachers’ overall CRT sample mean was lower than that of a normative 
group of British non-teacher engineering supervisors and junior managers (33.8 vs 37.5, 
respectively) which have recently become available. The tertiary education teachers’ lower 
mean CRT scores raises some concerns as presumably it is these individuals who have to 
be able to pass on via their curricula and teaching, the critical thinking “spirit” that is 
needed as a prerequisite for their students to learn ‘good’ thinking. As expected, there were 
no significant differences found between the mean CRT scores of female and the male 
teachers in the sample. There were also no significant differences between mean CRT 
scores for four age ranges tested. 
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The metacognitive skills or abilities (eg. drawing inferences based on information) and the 
dispositions (eg. drawing conclusions cautiously) reflected in the CRT and other similar 
critical thinking tests are presumed to be leamable. The question, therefore, is why then are 
they not reflected in the comparative outcomes reported about the normative data, stage of 
course and graduate vs non-graduate data. There may be many explanations for the failure 
to find hypothesised between-group differences. Some of the more important ones may be 
that firstly, the CRT does not measure the kind of critical reasoning that tertiary teachers 
have and apply and/or critical thinking may be context specific. Secondly, the sample of 
tertiary teachers may have been somehow not representative of what would be found using 
other teachers or a larger sample. Thirdly, the teachers in the sample did not command a 
level of critical reasoning beyond the comparative manager normative sample or up to the 
level of the engineering normative sample. Finally, critical reasoning may not have been 
taught or effectively taught during the university courses undertaken. Perhaps their 
lecturers do not have advanced critical thinking skills; maybe these skills and dispositions 
are taught but not effectively learned or perhaps, they are not applied or transferred to the 
CRT scenarios. 

Nevertheless, there is a fairly extensive body of literature now available which suggest that 
some knowledge and skills an individual has may serve to help, hinder or have no 
observable effect on the acquisition of new knowledge and skill (eg., Chi, Glaser & Farr, 
1988). The CRT does attempt to assess reasonably integrated and basic skills such as 
deriving documentary information, identifying assumptions, assessing the worth and 
applicability of new information, testing cautiously inferences made against provided 
information or the lack of it and so on. These are the sorts of integrated broad-based skills a 
tertiary education teacher might be expected to have, a university expected to teach and a 
student to learn. An assertion could be made that critical thinking is actually based on a 
range of logical inferences not defined and tested by the CRT. Nevertheless, it is worthy of 
note that the CRT was constructed to examine a range of critical reasoning skills and 
dispositions all of which are linked to or developed from earlier research in this area. 

A psychometric test such as the CRT, undoubtedly has its problems but it is also possible 
that, at least in some university courses, students’ critical reasoning is not being further 
measurably developed. The evidence from this study, given the CRT’s basic validity, can 
be used to support this view point. A reasonable overall conclusion is that the tertiary 
teachers in the present research, may not be being effectively taught or are not learning the 
generic critical thinking skills measured by the CRT and other similar psychometric tests. 
University curricula and teaching, especially in education, should reflect the planned 
development of critical reasoning within a domain or discipline and its application to the 
workplace. As well, university teachers should, given that they use critical reasoning 
themselves, try to make these generic critical reasoning skills and dispositions quite explicit 
during their teaching. They should then attempt to assess these just as readily as they 
assess specific subject matter content. It is this writer’s contention, based on consideration 
of the CRT, the results of this study and typical subject matter taught to tertiary education 
students (as well as how it is taught), that many teachers need help through staff 
development with the development and quite explicit use in teaching of critical reasoning. 
This is the case especially with those more generic aspects of critical reasoning which 
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might lead, at least on some occasions, to positive student adaptation during their further 
tertiary education and at work. 

Generic, metecognitive skills and dispositions relevant to disciplines which underpin 
professional expertise appear to be basic to individual vocational adaptability and progress. 
This adaptability and the critical reasoning which underpin it are not only basic to an 
individual’s professional contribution but also in the end, important for continued national 
development. If this small contribution to research in this area suggests anything it is that 
the field of critical reasoning is one which requires a lot more research, with a range of 
methodologies and measures, especially in the area of tertiary education. Critical 
reasoning certainly is emerging as a specific area of university student learning which 
needs to be addressed and more effectively taught and assessed in the new millennium. 
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